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© a video camera (10) includes a motion detecting 
circuit (20) in which correlative values of pixels at a 
present field are evaluated through the comparison 
of an image signal of a last field and an image signal 
of the present field. In the motion detecting circuit 
(20), an average correlative value and a minimum 
correlative value are further evaluated. A microcom- 
puter (26) determines whether or not these correl- 
ative values satisfy predetermined conditions for 
each of detection areas defined within an image field 
(44). A detection area which satisfies all the pre- 
determined conditions becomes a valid detection 
area. A whole motion vector is detected by methods 
different from each other according to the number of 
the valid detection. 
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BACKGROUND OF THE INVENTION 

Field of the invention 

The present invention relates to a video cam- 
era. More specifically, the present invention relates 
to a compact video camera with electronic picture 
stabilizer, which is utilized as a camcorder, for 
example. 

Description of the prior art 

One example of a method for detecting an 
unintentional motion component of an image sens- 
ing device is disclosed in, for example, the 20th 
image engineering conference in 1989 by Mat- 
sushita Electric Industrial Corporation. In this prior 
art method, by utilizing a motion vector obtained by 
a representative point matching method disclosed 
in, for example, Japanese Patent Application Lay- 
ing-open No. 61(1986)-201581 [H04N 7/137], the 
motion component of the image sensing device is 
detected according to image information. In this 
prior art, an electronic picture stabilization is per- 
formed on the basis of a whole motion vector 
obtained from the image information. 

More specifically, in this prior art, four detec- 
tion areas are arranged in an image field, and four 
portion motion vectors are obtained from the image 
field. Then, the whole motion vector is evaluated by 
averaging the portion motion vectors of the four 
detection areas, or the whole motion vector is eval- 
uated by averaging portion motion vectors of two 
portion motion vectors having intermediate values 
of the four portion motion vectors, whereby the 
electronic picture stabilizer is performed on the 
base of the whole motion vector thus evaluated. 

However, in such a method, in a case where an 
object passed through a detection area, for exam- 
ple, when there is a detection area in which a 
portion motion vector which is not caused by the 
unintentional motion of the camera is detected, the 
whole motion vector is affected by this detection 
area, and therefore, the motion of the camera can 
not be accurately detected, and thus, the electronic 
picture stabilizer can not be performed with preci- 
sion. 

SUMMARY OF THE INVENTION 

Therefore, a principal object of the present 
invention is to provide a video camera having a 
novel electronic picture stabilizer. 

Another object of the present invention is to 
provide a video camera with electronic picture sta- 
bilizer capable of accurately performing the elec- 
tronic picture stabilization. 
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In a video camera according to the present 
invention, valid detection area determinating means 
includes first means for evaluating correlative val- 
ues of pixels at a present field or frame on the 

5 basis of an image signal of a last field or frame and 
an image signal of the present field or frame; 
means for evaluating an average correlative value 
that is a mean value of the correlative values; 
means for evaluating a minimum correlative value 

70 that is a minimum value out of the correlative 
values; means for evaluating a value obtained by 
dividing the average correlative value by the mini- 
mum correlative value; means for evaluating a gra- 
dient associated with the minimum correlative val- 

75 ue; and means for determined whether or not a 
motion vector is correctly detected in accordance 
with whether or not all of the following conditions 
are satisfied: 

(A) the average correlative value > a first thresh- 
20 old value; 

(B) the value obtained by dividing the average 
correlative value by the minimum correlative val- 
ue > a second threshold value; 

(C) the gradient > a third threshold value (when 
25 the average correlative value ^ a fifth threshold 

value); and 

(D) the gradient > a fourth threshold value (when 
the average correlative value < the fifth thresh- 
old value), the third threshold value being equal 

30 to or larger than the fourth threshold value (the 
third threshold value ^ the fourth threshold val- 
ue). 

As indicated in the above described conditions 
(C) and (D), the threshold values to be compared 

35 with the gradient are changed in accordance with 
the average correlative value. When a contrast in a 
screen is large, that is, when the average correl- 
ative value is large, the threshold value to be 
compared with the gradient is set to be large as 

40 indicated by the condition (C), and when the con- 
trast in the screen is small, that is, when the 
average correlative value is small, the threshold 
value to be compared with the gradient is set to be 
small as indicated by the condition (D). Therefore, 

45 the gradient can be compared with the threshold 
value in corresponding to a change of the contrast 
in the screen (the third threshold value or the fourth 
threshold value), and therefore, determination 
whether or not an object having repeated pattern 

so exists in the image field becomes to be performed 
more correctly. 

Then, it is determined whether or not each of 
respective detection areas defined within the 
screen image field is a valid detection area by the 

55 valid detection area determinating means through 
determination whether or not all of the above de- 
scribed conditions are satisfied in respect to the 
detection area. 
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In a case where the number of the valid detec- 
tion areas is larger than a predetermined threshold 
value, first, a first absolute value and a second 
absolute value are evaluated for each detection 
area by first calculation means and second calcula- 
tion means, respectively. By adding the first ab- 
solute value and the second absolute value to each 
other for each detection area by addition means so 
as to obtain a first dispersion, and the arbitrary 
number of the first dispersions are selected by 
selection means in order of small. A whole motion 
vector is detected on the basis of portion motion 
vectors of the detection areas representing the 
selected first dispersions. 

When the number of the valid detection areas 
is less than the predetermined value, first, a portion 
motion vector of an arbitrary invalid detection area 
is replaced with the whole motion vector of a field 
or frame before the present field or frame by first 
replacement means. Thereafter, the portion motion 
vector of the invalid detection area that is replaced 
with the whole motion vector and the portion mo- 
tion vectors of the valid detection areas are utilized 
for detecting the whole motion vector at the 
present field or frame according to the above de- 
scribed processing. 

Furthermore, the number of valid detection 
areas is zero, as the whole motion vector of the 
present field or frame, a resulted vector by mul- 
tiplying the whole motion vector evaluated one field 
or frame before by a predetermined coefficient less 
than 1 (decimal number less than 1) is utilized. 

In accordance with the present invention, since 
a detecting method for detecting the whole motion 
vector is changed according to the number of valid 
detection areas, even if a portion motion vector that 
is not caused by the unintentional motion of the 
camera is detected in performing the electronic 
image stabilization, the influence thereof becomes 
very small, and therefore, it becomes possible to 
accurately detect an unintentional motion amount. 
Therefore, the unintentional motion of the camera 
can be corrected with precision. 

It is correctly determined whether or not the 
object having repeated pattern (stripe or the like) 
exists in the image, and a detection area is hardly 
determined as an invalid detection area even when 
the contrast in the screen is slightly fallen, and 
therefore, a detection accuracy of the motion vec- 
tor is increased. 

Furthermore, the whole motion vector may be 
evaluated on the basis of a second dispersion as 
described in the following. More specifically, first, 
the second dispersion is evaluated by second dis- 
persion calculation means by utilizing an average 
vector of the portion motion vectors, that is evalu- 
ated by averaging means, and the portion motion 
vectors of the respective detection areas. A fact 



that the second dispersion is large indicates that 
variation of the portion motion vectors is large. This 
tendency occurs in a case where there is an object 
which moves in only a portion of the screen. On 

5 the other hand, a fact that the second dispersion is 
small means that the variation of the portion motion 
vectors is small. This tendency occurs in a case 
where no moving object exists on the screen, or a 
case where there is an object which moves all over 

10 the screen. 

Therefore, if the whole motion vector by utiliz- 
ing the portion motion vector of the case where the 
second dispersion is large, the whole motion vector 
having low reliability is obtained. In contrast, if the 

75 whole motion vector is evaluated by utilizing the 
portion motion vector of the case whether the sec- 
ond dispersion is small, the whole motion vector 
having high reliability is obtained. Therefore, in the 
case where the second dispersion is large, a value 

20 of the whole motion vector is made small by mul- 
tiplying the average vector of the portion motion 
vectors by a small coefficient. Furthermore, in the 
case where the second dispersion is small, a value 
of the whole motion vector is made large by mul- 

25 tiplying the average vector of the portion motion 
vectors by a large coefficient. That is, the whole 
motion vector of the case of the large second 
dispersion is not used as possible, and in contrast, 
the whole motion vector of the case of the small 

30 second dispersion is used as possible. 

In addition, a degree of a magnitude of the 
second dispersion is determined by comparing 
means, and the whole motion vector is detected in 
accordance with a comparison result. 

35 In accordance with the present invention, by 

grasping a status of the screen through evaluation 
of the second dispersion, the whole motion vector 
is detected in accordance with the magnitude of 
the second dispersion, and therefore, it is possible 

40 to more correctly detect the whole motion vector. 
Accordingly, the unintentional motion amount can 
be detected with precision. 

The above described objects and other ob- 
jects, features, aspects and advantages of the 

45 present invention will become more apparent from 
the following detailed description of the present 
invention when taken in conjunction with the ac- 
companying drawings. 

50 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram showing one em- 
bodiment according to the present invention; 
Figure 2 is a block diagram showing a motion 
55 detecting circuit of Figure 1 embodiment; 

Figure 3 is an illustrative view showing a princi- 
ple of an electronic zooming and detection areas 
within an image field; 
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Figure 4 is an illustrative view showing a princi- 
ple of the electronic zooming and representative 
points and sampling points in the detection 
areas; 

Figure 5 is an illustrative view showing a method 
for detecting an object having repeated pattern 
by utilizing a pixel having a minimum correlative 
value and four pixels around the same; 
Figure 6 is an illustrative view showing a princi- 
ple of an electronic picture stabilization; 
Figure 7 is an illustrative view showing the de- 
tection areas within the image field, to which a 
representative point matching method is applied; 
Figure 8 is a flowchart showing an operation of 
the embodiment; 

Figure 9 is a flowchart showing an operation 
succeeding to Figure 8; 

Figure 10 is a flowchart showing a modified 
example of an operation for determining a whole 
motion vector; and 

Figure 1 1 is a graph showing a change of correl- 
ative values in respect to coordinate positions 
for each image status in a case where the 
representative point matching method is utilized. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

A video camera 10 of a preferred embodiment 
shown in Figure 1 includes a solid-state image 
sensing device 12 such as a CCD (Charge-Coupled 
Device) which converts an optical signal being in- 
putted from an object (not shown) through a lens 
14 into an electric signal. The electric signal from 
the solid-state image sensing device 12 is inputted 
to a camera circuit 16. As well known, the camera 
circuit 16 includes a sample-hold circuit by which 
the electric signal from the solid-state image sens- 
ing device 12 is sampled and held. A level of the 
electric signal thus sampled and held is adjusted 
by an AGC (Automatic Gain Control), and synchro- 
nization signals are added to the electric signal by 
a synchronization signal adding circuit (not shown). 
Thus, the camera circuit 16 converts the electric 
signal from the solid-state image sensing device 12 
into an analog video signal. The analog video sig- 
nal is further converted into a digital video signal 
by an A/D converter 18. The digital video signal is 
applied to a motion detecting circuit 20. As the 
motion detecting circuit 20, for example, an LSI 
"L7A0948" manufactured by Sanyo Electric Co., 
Ltd. who is an assignee of the present invention 
may be utilized. Under control of a memory control 
circuit 22 which is included in the same LSI con- 
stituting the motion detecting circuit 20, the digital 
video signal is written into a field memory 24 field 
by field. 
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The motion detecting circuit 20 evaluates, for 
each of four detection areas A, B, C and D shown 
in Figure 3, a position of one point having a highest 
correlation degree (a minimum correlative value) 

5 and positions of four points around the one point, 
and correlative values by utilizing a well-known 
representative point matching method. The position 
data and the correlative value data are applied to a 
microcomputer 26. 

w More specifically, with referring Figure 2, the 

motion detecting circuit 20 shown in Figure 1 in- 
cludes an input end 28 which receives the digital 
video signal from the A/D converter 18. The digital 
video signal inputted to the input end 28 is applied 

75 to the representative point memory 32 and a sub- 
tracting circuit 34, respectively, through a filter 30. 
The filter 30 is a kind of digital low-pass filter which 
is utilized for improvement of an S/N ratio so as to 
secure a significant detection accuracy with a less- 

20 er number of representative points. The representa- 
tive point memory 32 stores position data and 
luminance data of a plurality of representative 
points within each of the respective detection areas 
A-D shown in Figure 3. In this embodiment shown, 

25 each of the detection areas is divided into thirty 
(30) regions, and therefore, thirty (30) representa- 
tive points are determined, and accordingly, the 
representative point memory 32 stores the position 
data and the luminance data of the thirty (30) 

30 representative points. Each of the divided regions 
42 (Figure 4) is constituted by 32 pixels in a 
horizontal direction (X direction) x 16 pixels in a 
vertical direction (Y direction). 

The subtracting circuit 34 executes subtracting 

35 operations of the luminance data of the representa- 
tive point of the last field read-out the representa- 
tive point memory 32 and luminance data of all the 
pixels of the present field applied from the input 
end 28 via the filter 30, and obtains absolute values 

40 of subtraction results. That is, the subtracting cir- 
cuit 34 evaluates a luminance difference between 
the luminance data of the present field and the 
luminance data of the last field, and applies the 
luminance differences to an accumulating and add- 

45 ing circuit 36. The accumulating and adding circuit 
36 executes an accumulation and addition of the 
luminance differences of thirty (30) in this embodi- 
ment obtained by the subtracting circuit 34 of the 
same position or pixel in the same region 42 so as 

50 to output correlative values data. The correlative 
values data is applied to an arithmetic operation 
circuit 38 which evaluates a minimum correlative 
value and calculates an average correlative value 
for each of the detection areas A-D, and evaluates 

55 position data of the pixel having the minimum cor- 
relative value. Data of the minimum correlative val- 
ue, average correlative value and positions thus 
obtained by the arithmetic operation circuit 38 is 

4 
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applied to the above described microcomputer 26 
from an output end 40. In addition, such arithmetic 
operations for the correlative values can be per- 
formed by the above described LSI "L7A0948". 

Then, in the microcomputer 26, a motion vec- 
tor of a whole of a screen, i.e. the image field 44 
(Figure 3) (simply called as "whole motion vector") 
is calculated on the basis of the position data and 
the correlative value data. 

First, a deviation of a pixel indicative of the 
minimum correlative value from the representative 
point is evaluated on ihe basis of the position data 
of that pixel, and the deviation is made as a portion 
motion vector. In addition, in order to make a 
detection accuracy of the portion motion vector 
good, an internal interpolation by utilizing the cor- 
relative values of the four pixels around the pixel 
having the minimum correlative value so as to 
calculate the position data of the pixel having the 
minimum correlative value. 

In addition, the microcomputer 26 further evalu- 
ates a propriety of the portion motion vector de- 
tected for each detection area, that is, determines 
whether each of the detection areas is a valid 
detection area or an invalid detection area on the 
basis of the status of the image. In this embodi- 
ment shown, since the representative point match- 
ing method is utilized, the change of the correlative 
value with respect to the coordinate positions be- 
comes similar to Figure 1 1 (described later). 

Now, in order to determine whether or not the 
contrast of the screen is low, whether or not there 
is a moving object in the detection area, and 
whether or not there is an object having repeated 
pattern (stripes, for example) in the detection area, 
conditions (A) - (C) are defined as follows. 

(A) average correlative value > a ; 

(B) value obtained by dividing average correl- 
ative value by minimum correlative value > 0 ; 

(C) gradient > 7 (when average correlative value 
^ e ); and 

(D) gradient > 5 (when average correlative value 

<o 

In addition, a , £ , 7 , 5 , and e are constant 
threshold values, and 7^5. For example, the 
threshold values are set as a = 36, /3 = 7, 7 = 8, 
5 = 4, and e = 128. 

The microcomputer 26 determines whether or 
not the contrast of the screen is low in accordance 
with the above described condition (A). The micro- 
computer 26 further determines whether or not the 
moving object exists within the detection area on 
the basis of the above described condition (B). 
Furthermore, the microcomputer 26 determines 
whether or not the object having repeated pattern 
exists in the detection area on the basis of the 
above described condition (C) at a timing that the 
average correlative value is equal to or larger than 
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the threshold value e , or the above described 
condition (D) at a time that the average correlative 
value is smaller than the threshold value e . These 
processings are executed for each of the detection 

5 areas A - D so that it is determined whether or not 
the portion motion vector of each of the detection 
areas A - D is erroneously detected due to the 
moving object or the like other than the uninten- 
tional motion and thus reliable, that is, whether or 

io not each of the detection areas A - D is the valid 
detection area. If a detection area satisfies all the 
above described conditions (A) - (D), the detection 
areas is the valid detection area, and if any one of 
the above described conditions is not satisfied, the 

75 detection area is determined as an invalid detection 
area. 

In concrete, whether or not the detection area 
is the valid detection area is determined as follows: 
At first, when the contrast of the screen is low, 

20 the luminance difference is small, and therefore, 
the correlative value becomes small. When a whole 
of the screen is white, for example, the correlative 
value is small. In such a case, the reliability of the 
detection result is small, and therefore, only when 

25 the condition (A) is satisfied, that is, the average 
correlative value > a , the detection area is deter- 
mined as the valid detection area. In addition, the 
threshold value a can be determined through field 
tests or examinations. Thus, on the basis of the 

30 average correlative value, it is determined whether 
or not the screen is low contrast. 

Furthermore, when the moving object exists in 
the detection area, the correlative value at a portion 
occupied by the moving object and the correlative 

35 value no moving object exists are different from 
each other. Various kinds of correlative values are 
obtained by the portion occupied by the moving 
object, and the correlative value from that portion 
becomes a large value generally (the correlation 

40 degree becomes low). Therefore, when the moving 
object exists within the detection area, there is a 
possibility that the minimum correlative value be- 
comes large, and the portion motion vector of that 
detection area may be erroneously detected. If the 

45 portion motion vector is erroneously detected, the 
whole motion vector is also erroneously detected; 
however, when the average correlative value is 
large, the portion motion vector is reliable even if 
the minimum correlative value is large at some 

so extent. On the other hand, when the average correl- 
ative value is small, the portion motion vector is 
reliable only when the minimum correlative value is 
smaller. Therefore, in concrete, the detection area 
is determined as the valid detection area when (the 

55 average correlative value / the minimum correlative 
value) > 7, and if this condition is not satisfied, the 
portion motion vector of the detection area is not 
utilized so as to avoid the influence due to the 

5 
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above described erroneous detection. Thus, the 
microcomputer 26 determines the presence or ab- 
sence of the moving object by evaluating the value 
of the average correlative value / the minimum 
correlative value. 

Furthermore, in the microcomputer 26, one 
point having the minimum correlative value and the 
correlative values of four points around the one 
point are utilized so as to detect the object having 
repeated pattern (stripes, for example). 

More specifically, on the assumption that the 
minimum correlative value is M, and the correlative 
values of the four points at left, right, up, and down 
are L, R, U, and D as shown in Figure 5. The 
differences between the respective correlative val- 
ues, i.e. L - M, R - M, U - M and D - M are 
calculated, and a minimum value of the differences 
is defined as the gradient. When the average cor- 
relative value is equal to or larger than the thresh- 
old value e , the gradient is compared with the 
threshold value -y that is determined through field 
tests. Then, if the gradient is larger than the thresh- 
old value 7 , it is determined that the detection 
area is the valid detection area, and if the gradient 
is equal to or smaller than the threshold value -y , 
the detection area is determined as the invalid 
detection area. On the other hand, when the aver- 
age correlative value is smaller than the threshold 
value e , the gradient is compared with the thresh- 
old value 5 that is determined through field tests. If 
the gradient is larger than the threshold value 5 , 
the detection area is determined as the valid detec- 
tion area, and if the gradient is equal to or smaller 
than the threshold value 5 , it is determined that the 
detection area is the invalid detection area. 

Thus, in accordance with the conditions (A) - 
(D), it is determined whether or not each of the 
detection areas is the valid detection area. 

Then, in accordance with the number of valid 
detection areas, the motion amount between the 
fields, i.e. the whole motion vector is determined. 
Therefore, the whole motion vector is representa- 
tive of the motion amount between the fields and a 
direction thereof. 

The whole motion vector thus evaluated is ap- 
plied to the memory control circuit 22. In the mem- 
ory control circuit 22, a start address for reading- 
out the field memory 24 is determined on the basis 
of the whole motion vector, and therefore, the digi- 
tal video signal stored in the field memory 24 
becomes to be read-out at the start address. That 
is, the memory control circuit 22 moves an extract- 
ing area 46 (Figure 6) formed by the digital video 
signal from the field memory 24 in accordance with 
the whole motion vector calculated by the micro- 
computer 26. 

In addition, since the extracting area 46 can not 
be moved by the digital video signal read-out from 
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the field memory 24 as it is, an electronic zooming 
circuit 48 (Figure 1) is utilized. 

With referring Figure 6, the electronic zooming 
circuit 48 (Figure 1) defines the image extracting 

5 area 46 wherein an image is enlarged according to 
a zooming magnification with respect to the image 
field 44. A position of the image extracting area 46 
can be freely moved within a range of the image 
field 44 by changing the start address for reading- 

70 out the digital video signal from the field memory 
24. Then, in order to obtain a video signal for a 
whole area of the image field 24 on the basis of the 
digital video signal extracted from the image ex- 
tracting, area 46, an image is enlarged by utilizing 

75 an internal interpolation on the basis of the digital 
video signal read-out the field memory 24. 

Thus, by zooming-up an image of an arbitrary 
image extracting area 46 within the image field 44 
in an electronic manner by the electronic zooming 

20 circuit 48 (Figure 1), a correctable range 50 that is 
equal to a difference between the image field 44 
and the image extracting area 46 can be formed. 

If an unintentional motion occurs in the video 
camera 10 as shown in Figure 7 according to a 

25 vibration of a hand of a person who operates the 
video camera 10, an image from the video camera 
10 is blurred, and resulting in a case where an 
object person exists in a left-lower portion within 
the image field 44 (shown at an upper portion in 

30 Figure 7) or a case where an object person exists 
at a right-upper portion within the image field 
(shown at a lower portion in Figure 47. Therefore, 
by moving the image extracting area 46 at every 
field according to the whole motion vector that is 

35 calculated by the microcomputer 26, as shown at a 
right portion in Figure 7, the object person can be 
just positioned in the image extracting area 46. 

The digital video signal thus outputted from the 
electronic zooming circuit 48 is converted into an 

40 analog signal by a D/A converter 52 so as to be 
outputted from an output terminal 54. 

In addition, the processings set forth in the 
following may be executed by the microcomputer 
26. 

45 At first, the microcomputer 26 evaluates a 

mean value of four portion motion vectors (average 
vector) and a mean value of portion motion vectors 
of the valid detection areas, respectively. 

Then, a dispersion is calculated by the micro- 

50 computer 26. A degree of variation of the portion 
motion vector of each of the detection areas is 
evaluated according to the dispersion. The disper- 
sion is represented by the following equations. 

55 X direction dispersion = E (| X direction portion 
motion vectors of detection areas - X direction 
average vector] ) / the number of the detection 
areas; and 

6 



1 1 



EP 0 631 432 A1 



12 



Y direction dispersion = E (| Y direction portion 
motion vectors of detection areas - Y direction 
average vector] ) / the number of the detection 
areas 

wherein, the X direction portion motion vector in- 
dicates an X direction component of the portion 
motion vector, and the Y direction portion motion 
vector indicates a Y direction component of the 
portion motion vector, and the X direction average 
vector indicates an X direction component of the 
average vector, and the Y direction average vector 
indicates a Y direction component of the average 
vector. 

in accordance with the dispersion as described 
later, a dispersion coefficient Hk is determined, and 
by multiplying the dispersion coefficient Hk by the 
mean value of the portion motion vectors of the 
valid detection areas, the whole motion vector is 
calculated. In addition, it is needless to say that the 
dispersion coefficient Hk is determined in the X 
direction and the Y direction, respectively, and the 
X direction dispersion coefficient is multiplied to 
the X direction component of the means value of 
the portion motion vectors of the valid detection 
areas, and the Y direction dispersion coefficient is 
multiplied to the Y direction component of the 
mean value of the portion motion vectors. 

(1) 0^ dispersion < 8: Hk = 1 

(2) 8 ^ dispersion < 16: Hk = 0.75 

(3) 16 ^ dispersion < 24: Hk = 0.5 
(2) 24 ^ dispersion < 32: Hk = 0.25 

In addition, in a case where the dispersion 
exceeds "32", a minimum one of the four portion 
motion vectors is defined as the whole motion 
vector. 

Thus, if there is a valid detection area, the 
whole motion vector is calculated by multiplying 
the dispersion coefficient Hk to the mean value of 
the portion motion vectors of the valid detection 
areas; however, there is no valid detection area, the 
whole motion vector is defined as "the whole mo- 
tion vector of the present field x a coefficient less 
than 1". In addition, as the coefficient less than 1, 
in the embodiment shown, "0.97" is utilized. 

In addition, a reason why the dispersion coeffi- 
cient is set stepwise in accordance with the mag- 
nitude of the dispersion is as follows: 

A fact that the dispersion is large indicates that 
the variation of the each of respective portion mo- 
tion vectors is large. This tendency occurs in a 
case where a moving object exists at only a portion 
of the screen. On the other hand, a fact that the 
dispersion is small indicates that the variation of 
each of the respective portion motion vectors is 
small. This tendency occurs in a case where no 
moving object exists in the screen, or a case where 
a moving object which moves all over the screen 
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exists. 

Therefore, if the whole motion vector is evalu- 
ated by utilizing the portion motion vectors of a 
case where the dispersion is large, the whole mo- 

5 tion vector having low reliability is obtained. On the 
other hand, if the whole motion vector is evaluated 
by utilizing the portion motion vectors of a case 
where the dispersion is small, the whole motion 
vector having high reliability is obtained. Accord- 

10 ingly, when the dispersion is large, the whole mo- 
tion vector is made smaller by multiplying a small 
dispersion coefficient Hk to the mean value of the 
portion motion vectors. Furthermore, when the dis- 
persion is small, by multiplying a large dispersion 

75 coefficient Hk to the mean value of the portion 
motion vectors, the whole motion vector is made 
larger. That is, the whole motion vector of the case 
where the dispersion is large is not used as possi- 
ble, and in contrast, the whole motion vector of the 

20 case where the dispersion is small is used as 
possible, so that the whole motion vector having 
high reliability can be obtained. 

By utilizing the whole motion vector thus ob- 
tained, in accordance with the above described 

25 manner, the unintentional motion is corrected, that 
is, the electronic picture stabilization is performed. 

With referring Figure 8 to Figure 10, a major 
operation in the microcomputer 26 of the video 
camera 10 will be described. In addition, in a 

30 flowchart of Figure 8, "n" ("1" to "4") is cor- 
responding to the detection area A to the detection 
area D. 

In a step S1 shown in Figure 8, "n" of the 
detection area is initially set as "1". If the average 

35 correlative value of the detection area n is larger 
than the threshold value a in a step S3, the pro- 
cess proceeds to a step S5 wherein it is deter- 
mined whether or not the value obtained by divid- 
ing the average correlative value by the minimum 

40 correlative value is larger than the threshold value 
£ . If the value is larger than the threshold value £ 
in the step S5, in a step S7, it is determined 
whether or not the average correlative value is 
equal to or larger than the threshold value e . If the 

45 average correlative value is equal to or larger than 
the threshold value e , in a step S9, it is determined 
whether or not the gradient of the detection area n 
is larger than the threshold value 7 . If "YES" is 
determined in the step S9, in a step S13, the 

50 detection area n is determined as a valid detection 
area. On the other hand, if the average correlative 
value is smaller than the threshold value e in the 
step S7, in a step S11, it is determined whether or 
not the gradient of the detection area n is larger 

55 than the threshold value 5 . If "YES" is determined 
in the step S11, in a step S13, the detection area n 
is determined as a valid detection area. 
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At a time that the averaged correlative value of 
the detection area n is equal to or smaller than the 
threshold value a in the step S3, a time that the 
value obtained by dividing the average correlative 
value of the detection area n by the minimum 
correlative value is equal to or smaller than the 
threshold value & in the step S5, a time that the 
gradient of the detection area n is equal to or 
smaller than the threshold value 7 in the step S9, 
or a time that the gradient of the detection area n is 
equal to or smaller than the threshold value 5 in the 
step S11, the process proceeds to a step S15 in 
which, the detection area n is determined as an 
invalid detection area. 

After the step S13 or the step S15, in a step 
S17, "n" indicative of the detection area is incre- 
mented by 1. Then, in a step S19, it is determined 
whether or not "n" is equal to or smaller than "4", 
and if "YES", the process returns to the step S3. 
That is, the above described processings are re- 
peated until "n" exceeds "4" in the step S19. If 
"n" exceeds "4" in the step S19, the process 
proceeds to a step S21 shown in Figure 9. 

In the step S21, the number of the valid detec- 
tion areas is counted. In a step S23, it is deter- 
mined whether or not the number of the valid 
detection areas is "4", and if "NO", in a step S25, 
it is determined whether or not the number of the 
valid detection areas is "3". If "YES" is determined 
in the step S23 or the step S25, that is, if the 
number of the valid detection areas is more than 
"3", in a step S27, mean values are calculated of 
the X (horizontal, for example) direction portion 
motion vectors and the Y (vertical, for example) 
direction portion motion vectors are calculated for 
each of the valid detection areas. Then, in a step 
S29, by utilizing the mean values calculated in the 
step S27, an absolute value of the X direction and 
an absolute value of the Y direction are evaluated 
for each of the valid detection areas, and then, the 
X direction absolute value and the Y direction ab- 
solute value are added to each other for each 
detection area to obtain the dispersion. The X 
direction absolute value is an absolute value of a 
difference between the X direction portion motion 
vectors of the respective detection areas and the 
mean value of the X direction portion motion vec- 
tors of the valid detection areas. Furthermore, the Y 
direction absolute value is an absolute value of a 
difference between the Y direction portion motion 
vectors of the respective detection areas and the 
mean value of the Y direction portion motion vec- 
tors of the valid detection areas. Then, in a step 
S31, two dispersions having smaller values are 
selected from the four dispersions calculated in the 
step S29, and a mean value of the portion motion 
vectors of the detect ion areas corresponding to 
the selected dispersions is calculated. Then, in a 



step S33, the mean value is used as the whole 
motion vector. 

On the other hand, if "NO" is determined in the 
step S25, in a step S35, it is determined whether or 

5 not the number of the valid detection areas is "2". 
If the number of the valid detection areas is "2", in 
a step S37, the portion motion vector of an ar- 
bitrary one of the invalid detection areas is re- 
placed with the whole motion vector evaluated be- 

70 fore one field, and then, the process proceeds to 
the step S27. Then, through the execution of the 
steps S27 - S33, the whole motion vector is deter- 
mined. In this embodiment shown, the portion mo- 
tion vector of one of the two invalid detection area 

75 is replaced by the whole motion vector of the last 
field, but the portion motion vector of the remaining 
invalid detection area is not replaced with the 
whole motion vector. Therefore, the whole motion 
vector is determined on the basis of the portion 

20 motion vectors of the two valid detection areas and 
the portion motion vector of the invalid detection 
area being replaced with the whole motion vector. 

If it is determined that the number of the valid 
detection areas is not "2" in the step S35, in a step 

25 S39, it is determined whether or not the number of 
the valid detection areas is "1". If "YES" is deter- 
mined in the step S39, in a step S41, the portion 
motion vectors of arbitrary two invalid detection 
areas are replaced with the whole motion vector 

30 evaluated one field before and the whole motion 
vector evaluated two fields before, respectively, 
and the process proceeds to the step S27. Then, 
through the execution of the steps S27 - S33, the 
whole motion vector can be evaluated. In this em- 

35 bodiment shown, the portion motion vectors of the 
two invalid detection areas out of the three invalid 
detection areas are replaced with the whole motion 
vector evaluated one field before and the whole 
motion vector evaluated two field before, respec- 

40 tively, and for the remaining one invalid detection 
area, the portion motion vector is not replaced with 
the whole motion vector. Therefore, in this embodi- 
ment shown, the whole motion vector is determined 
on the basis of the portion motion vector of one 

45 valid detection area and the portion motion vectors 
of the two invalid detection area being replaced 
with the whole motion vectors . 

If the number of the valid detection areas is not 
"1", that is, it is determined that all the detection 

50 areas are invalid detection areas in the step S39, in 
a step S43, by multiplying the coefficient "0.97" to 
the whole motion vector evaluated one field before, 
the whole motion vector of the present field in the 
step S33 is obtained. 

55 Thus, the unintentional motion is corrected by 

utilizing the whole motion vector according to the 
number of the valid detection areas; however, a 
correction method itself is well known, and there- 
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fore, a detailed description thereof is omitted here. 

In accordance with this embodiment, the deter- 
mination whether or not there is an object having 
repeated pattern within the detection areas can be 
made correctly, and the detection area may not be 
determined as the invalid detection area even if the 
contrast of the screen is slightly lowered. There- 
fore, the validation or the invalidation of the detec- 
tion areas can be correctly determined, and ac- 
cordingly, the detection accuracy of the whole mo- 
tion vector becomes to be increased. 

That is, according to the above described em- 
bodiment, the disadvantages of the prior art set 
forth in the following can be eliminated. 

In the prior art method wherein the uninten- 
tional motion component is detected from the im- 
age information, the detection accuracy is largely 
affected by the status of the input image. There- 
fore, the propriety of the detected motion vector is 
evaluated by the microcomputer in accordance 
with the status of the image. 

Figure 1 1 is a graph showing a change of a 
correlative value with respect to coordinate posi- 
tions in a case where the representative matching 
method is utilized. Normally, as shown by a solid 
line a in Figure 11, the correlative value has a 
sharp minimum point indicative of the minimum 
correlative value, and the motion vector is deter- 
mined on the basis of the position of the minimum 
correlative value. In contrast, if the contrast of the 
screen is low, or if a moving object exists in the 
screen, as shown by a one-dotted line b or a 
dotted line c in Figure 11, the minimum correlative 
value can not be sharply detected, and therefore, 
the detection accuracy of the motion vector be- 
comes to be deteriorated. Furthermore, in a case 
where there is an object having repeated pattern 
(striped image) in the screen, it becomes very 
difficult to detect the motion vector. In order to 
determine the above described image status, in the 
prior art method, the following conditions are set 
with respect to the correlative value: 

(A) average correlative value > a ; 

(B) value obtained by dividing average correl- 
ative value by minimum correlative value > £ ; 
and 

(C) gradient > 7 

wherein, a , & and 7 are constant threshold values, 
and for example, the same are set as a = 36, £ = 
7 and 7 = 8. 

In the prior art method, for each of the detec- 
tion areas, it is determined whether or not the 
above described conditions (A) - (C) are satisfied. 
Then, if any one of the conditions is satisfied, the 
detection area is determined as an invalid detection 
area, and if all the conditions are satisfied, the 
detection area is determined as a valid detection 
area. Then, a motion vector is determined by utiliz- 
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ing the motion vectors obtained from the valid 
detection areas only. As a method for determinat- 
ing the motion vector, an averaging of the motion 
vectors of the valid detection areas is utilized, for 

5 example. 

However, in the above described conditions of 
the prior art method, if the contrast of the screen 
becomes to be slightly lowered, even when the 
condition (A) is satisfied, the condition (C) is not 

70 satisfied, and therefore, the threshold value 7 of the 
condition (C) is to be made smaller. However, the 
threshold value 7 is made smaller irrespective of 
the contrast of the screen, it may be erroneously 
determined whether or not there is an object hav- 

15 ing a repeating pattern in the screen, and therefore, 
the detection accuracy of the motion vector is 
dropped. In contrast, in the above described, em- 
bodiment, the gradient is compared with the large 
threshold value or the small threshold value ac- 

20 cording to another condition, such a problem can 
be solved. 

In addition, in the above described embodi- 
ment, the correlative value is calculated through 
accumulation and addition of the luminance differ- 
as ences; however, the correlative value may be eval- 
uated by utilizing a difference of the electric signals 
from the solid-state image sensing device 12 be- 
tween the adjacent fields (frames) instead of the 
luminance differences. 

30 Furthermore, in the step S31, only one disper- 

sion may be selected, that is, the arbitrary number 
of dispersions may be selected. 

Furthermore, the coefficient utilized in the step 
S43 is not limited to "0.97", and therefore, an 

35 arbitrary coefficient having a value larger than zero 
but smaller than 1 may be utilized. 

Furthermore, instead of the stops S29 and S31 
shown in Figure 9, processings shown in Figure 10 
may be executed. In this case, after the execution 

40 of the step S27 shown in Figure 9, the process 
proceeds to a step S45 shown in Figure 10. 

In the step S45, the dispersion is calculated. In 
calculating the dispersion, the X direction disper- 
sion and the Y direction dispersion are indepen- 

45 dently calculated as indicated in the above de- 
scribed equations. If 0 ^ dispersion < 8 in a step 
S47, in a step S49, the dispersion coefficient Hk is 
set as "1" (Hk = 1), and then, the process pro- 
ceeds to a step S51. If "NO" is detected in the 

50 step S47, in a step S53, it is determined whether or 
not 8 £ dispersion < 16. If 8 ^ dispersion < 16 in 
the step S53, in a step S55, the dispersion coeffi- 
cient Hk is set as "0.75" (Hk = 0.75), and then, 
the process proceeds to the step S51 . If "NO" is 

55 detected in the step S53, and if 16 £ dispersion < 
24 in a step S57, in a step S59, the dispersion 
coefficient Hk is set as "0.5" (Hk = 0.5), and then, 
the process proceeds to the step S51. If "NO" is 

9 



17 



EP 0 631 432 A1 



detected in the step S57, and if 24 £ dispersion < 
32 in a step S61 , in a step S63, the dispersion 
coefficient Hk is set as "0.25" (Hk = 0.25), and 
then, the process proceeds to the step S51. 

In the step S51, "a mean value of the portion 5 
motion vectors of the valid detection areas x Hk" is 
calculated. The process in the step S51 is per- 
formed for the X direction portion motion vectors 
and the Y direction portion motion vectors, respec- 
tively, and in the step S33, the result of the step w 
S51 for the X direction and the result of the step 
S45 for the Y direction are synthesized with each 
other, whereby the whole motion vector is ob- 
tained. 

On the other hand, "NO" is detected in the 75 
step S61, in a step S65, a portion motion vector 
having a minimum value is selected out of the four 
portion motion vectors, and the same is utilized as 
the whole motion vector in the step S33 shown in 
Figure 9. 20 

In the video camera 10 which operates as 
shown in Figure 10, even if moving objects exist in 
two detection areas out of the four detection areas 
such as a left half and a right half of the screen, 
that is, even if a moving object exists at only a 25 
portion of the screen, the influence due to the 
moving objects becomes small, and therefore, it is 
possible to determine the whole motion vector with 
precision. 

In addition, irrespective of the valid detection 30 
area or the invalid detection area, the whole motion 
vector which is obtained by multiplying the disper- 
sion coefficient Hk to the average value of the 
portion motion vectors of the detection areas. This 
becomes effective for a case where no determina- 35 
tion of the valid detection area or the invalid detec- 
tion area is performed. 

Furthermore, the dispersion may be compared 
with arbitrary values and the values of the disper- 
sion coefficients are not limited to the above de- 40 
scribed ones. 

Furthermore, the four detection areas are de- 
fined within the image field in the above described 
embodiments; however, the number of the detec- 
tion areas is arbitrary, and the larger number of the 45 
detection areas, the better the detection accuracy. 

Furthermore, in the above described embodi- 
ments, the luminance difference is evaluated be- 
tween the adjacent fields; however, the same may 
be evaluated between the adjacent frames. Further- 50 
more, the replacement of the whole motion vector 
and etc. may be performed not a unit of field but a 
unit of frame. 

Although the present invention has been de- 
scribed and illustrated in detail, it is clearly under- 55 
stood that the same is by way of illustration and 
example only and is not to be taken by way of 
limitation, the spirit and scope of the present inven- 



tion being limited only by the terms of the appen- 
ded claims. 

Claims 

1. A video camera, comprising: 

portion motion vector detecting means for 
detecting a portion motion vector of each of a 
plurality of detection areas formed in an image 
field; 

valid detection area determining means for 
determining whether or not each of the detec- 
tion areas is a valid detection area through 
determination whether or not each of the por- 
tion motion vectors is effective for detecting an 
unintentional motion of the camera; and 

whole motion vector detecting means for 
detecting a whole motion vector in accordance 
with methods different from each other in ac- 
cordance with the number of the valid detec- 
tion area determined by said valid detection 
area determining means. 

2. A video camera according to claim 1 , wherein 
said whole motion vector detecting means in- 
cludes 

first dispersion calculating means for ob- 
taining a first dispersion on the basis of the 
respective portion motion vectors, 

selection means for selecting the arbitrary 
number of the first dispersions having smaller 
values, and 

first detecting means for detecting the 
whole motion vector on the basis of the portion 
motion vectors of the detection areas corre- 
sponding to the selected first dispersions. 

3. A video camera according to claim 2, wherein 
said first dispersion calculating means includes 

first calculating means for evaluating a first 
absolute value representative of a difference 
between the portion motion vectors in a hori- 
zontal direction of each of the detection areas 
and a mean value of the portion motion vectors 
in the horizontal direction of the valid detection 
areas, 

second calculating means for evaluating a 
second absolute value representative of a dif- 
ference between the portion motion vectors in 
a vertical direction of each of the detection 
areas and a mean value of the portion motion 
vectors in the vertical direction of the valid 
detection areas, and 

adding means for obtaining said first dis- 
persion by adding the first absolute value and 
the second absolute value to each other for 
each of the detection areas. 
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4. A video camera according to claim 2 or 3, 
wherein said first detecting means detects the 
whole motion vector by averaging the portion 
motion vectors. 

5. A video camera according to any one of claims 
2 to 4, wherein the number of the detection 
areas is four, and said selection means selects 
two first dispersions. 

6. A video camera according to claim 1, wherein 
said whole motion vector detecting means in- 
cludes 

first replacement means for replacing the 
portion motion vector of the invalid detection 
area with the whole motion vector before the 
present field or frame when the number of 
valid detection areas is less than a predeter- 
mined threshold value, and 

second detection means for detecting the 
whole motion vector on the basis of the portion 
motion vector of the invalid detection area be- 
ing replaced with the whole motion vector and 
the portion motion vectors of the valid detec- 
tion areas. 

7. A video camera according to claim 6, wherein 
said first replacement means replaces the por- 
tion motion vector of the invalid detection area 
with the whole motion vector evaluated one 
field or one frame before. 

8. A video camera according to claim 1, wherein 
said whole motion vector detecting means in- 
cludes 

second replacement means for replacing 
the portion motion vectors of the invalid detec- 
tion areas with a plurality of whole motion 
vectors which are different from each other 
and evaluated before the present field or frame 
when the number of valid detection areas is 
less than a predetermined threshold value, and 

third detection means for detecting the 
whole motion vector on the basis of the portion 
motion vectors of the invalid detection areas 
being replaced with the whole motion vectors 
and the portion motion vector of the valid de- 
tection area. 



eludes fourth detection means for detecting the 
whole motion vector of the present field or 
frame by multiplying a predetermined coeffi- 
cient to the whole motion vector evaluated one 
5 field or frame before when the number of valid 

detection areas is zero. 

11. A video camera according to claim 10, wherein 
said coefficient is a numeral value smaller than 

70 1. 

12. A video camera according to claim 11, wherein 
said coefficient is 0.97. 

75 13. A video camera according to claim 1, wherein 
said valid detection area determinating means 
includes 

first means for evaluating correlative val- 
ues of pixels through comparison of an image 
20 signal of the last field or frame and an image 

signal of the present field or frame, 

means for evaluating an average correl- 
ative value that is a mean value of the correl- 
ative values, 

25 means for evaluating a minimum correl- 

ative value that is a minimum value of the 
correlative values, 

means for evaluating a value obtained by 
dividing the average correlative value by the 
30 minimum correlative value, 

means for evaluating a gradient associated 
with the minimum correlative value, and 

means for determining whether or not the 
portion motion vector is correctly detected in 
35 accordance with the following conditions: 

(A) average correlative value > first thresh- 
old value; 

(B) value obtained by dividing average cor- 
relative value by minimum correlative value 

40 > second threshold value; 

(C) gradient > third threshold value (when 
average correlative value Z fifth threshold 
value); and 

(D) gradient > fourth threshold value (when 
45 average correlative value < fifth threshold 

value), said third threshold value being 
equal to or larger than the fourth threshold 
value. 



9. A video camera according to claim 8, wherein so 
said second replacement means replaces the 
portion motion vectors of the invalid detection 
areas with the whole motion vectors evaluated 

one field or frame before and two fields or 
frames before. 55 

10. A video camera according to claim 1, wherein 
said whole motion vector detecting means in- 



14. A video camera according to claim 1, wherein 
said whole motion vector detecting means in- 
cludes 

second dispersion calculating means for 
obtaining a second dispersion on the basis of 
the portion motion vectors, 

comparing means for comparing the sec- 
ond dispersion with a predetermined threshold 
value, and 
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fifth detection means for detecting the 
whole motion vector by methods different from 
each other in accordance with a comparison 
result by the comparing means. 

15. A video camera according to claim 14, wherein 
said second dispersion calculating means in- 
cludes 

averaging means for evaluating an average 
vector that is a mean value of the portion 
motion vectors, and 

calculating means for evaluating the sec- 
ond dispersion by adding absolute values of 
differences between the portion motion vectors 
and the average vector to each other, and by 
dividing an addition result by the number of 
absolute values. 

16. A video camera according to claim 15, wherein 
said portion motion vector detecting means 
includes means for detecting X direction por- 
tion motion vectors and Y direction portion 
motion vectors, and said averaging means in- 
cludes means for evaluating an X direction 
average vector and a Y direction average vec- 
tor, respectively, and said calculating means 
evaluates the second dispersion of X direction 
component of the portion motion vector and 
the second dispersion of Y direction compo- 
nent of the portion motion vector. 

17. A video camera according to claim 14, wherein 
said comparison means compares a plurality 
of threshold values with the second dispersion. 

18. A video camera according to claim 14, wherein 
said fifth detecting means includes 

coefficient setting means for setting a dis- 
persion coefficient in accordance with a com- 
parison result by the comparing means, and 

means for obtaining the whole motion vec- 
tor by multiplying the dispersion coefficient to 
a mean value of the portion motion vectors of 
the valid detection areas. 

19. A video camera according to claim 14, wherein 
said fifth detecting means includes means for 
regarding a minimum portion motion vector as 
the whole motion vector when the second dis- 
persion is larger than a predetermined value. 

20. A video camera, comprising: 

first means for evaluating correlative val- 
ues of pixels through comparison of an image 
signal of the last field or frame and an image 
signal of the present field or frame, 

means for evaluating an average correl- 
ative value that is a mean value of the correl- 



ative values, 

means for evaluating a minimum correl- 
ative value that is a minimum value of the 
correlative values, 
5 means for evaluating a value obtained by 

dividing the average correlative value by the 
minimum correlative value, 

means for evaluating a gradient associated 
with the minimum correlative value, and 
w means for determining whether or not the 

portion motion vector is correctly detected in 
accordance with the following conditions: 

(A) average correlative value > first thresh- 
old value; 

75 (B) value obtained by dividing average cor- 

relative value by minimum correlative value 
> second threshold value; 

(C) gradient > third threshold value (when 
average correlative value ^ fifth threshold 

20 value); and 

(D) gradient > fourth threshold value (when 
average correlative value < fifth threshold 
value), said third threshold value being 
equal to or larger than the fourth threshold 

25 value. 

21. A video camera with electronic picture stabi- 
lizer which evaluates a whole motion vector by 
utilizing portion motion vectors of a plurality of 

30 detection areas defined within an image field, 

comparison: 

averaging means for evaluating an average 
vector that is a mean value of the portion 
motion vectors; 

35 dispersion calculating means for evaluating 

a dispersion by adding absolute values of dif- 
ferences between the portion motion vectors 
and the average vector to each other, and by 
dividing an addition result by the number of 

40 the absolute values; 

comparing means for comparing the dis- 
persion with a predetermined threshold value; 
and 

whole motion vector detecting means for 
45 detecting the whole motion vector in accor- 

dance with methods different from each other 
according to a comparison result by the com- 
paring means. 

50 
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